Image restoration under hazy weather condition, which is called single image dehazing, has been of significant interest for various computer vision applications. In recent years, deep learning-based methods have achieved success. However, existing image dehazing methods typically neglect the hierarchy of features in the neural network and fail to exploit their relationships fully. To this end, we propose an effective image dehazing method named Hierarchical Contrastive Dehazing (HCD), which is based on feature fusion and contrastive learning strategies. HCD consists of a hierarchical dehazing network (HDN) and a novel hierarchical contrastive loss (HCL). Specifically, the core design in the HDN is a Hierarchical Interaction Module, which utilizes multi-scale activation to revise the feature responses hierarchically. To cooperate with the training of HDN, we propose HCL which performs contrastive learning on hierarchically paired exemplars, facilitating haze removal. Extensive experiments on public datasets, RESIDE, HazeRD, and DENSE-HAZE, demonstrate that HCD quantitatively outperforms the state-of-the-art methods in terms of PSNR, SSIM and achieves better visual quality.
translated by 谷歌翻译
由于需要经济的储存和二元法规的效率,因此无监督的哈希对二元表示学习引起了很多关注。它旨在编码锤子空间中的高维特征,并在实例之间保持相似性。但是,大多数现有方法在基于多种的方法中学习哈希功能。这些方法捕获了数据的局部几何结构(即成对关系),并且在处理具有不同语义信息的实际特征(例如颜色和形状)的真实情况时缺乏令人满意的性能。为了应对这一挑战,在这项工作中,我们提出了一种有效的无监督方法,即共同个性化的稀疏哈希(JPSH),以进行二进制表示学习。具体来说,首先,我们提出了一个新颖的个性化哈希模块,即个性化的稀疏哈希(PSH)。构建了不同的个性化子空间,以反映不同群集的特定类别属性,同一群集中的自适应映射实例与同一锤子空间。此外,我们为不同的个性化子空间部署稀疏约束来选择重要功能。我们还收集了其他群集的优势,以避免过度拟合,以构建PSH模块。然后,为了在JPSH中同时保留语义和成对的相似性,我们将基于PSH和歧管的哈希学习纳入无缝配方中。因此,JPSH不仅将这些实例与不同的集群区分开,而且还保留了集群中的本地邻里结构。最后,采用了交替优化算法,用于迭代捕获JPSH模型的分析解决方案。在四个基准数据集上进行的大量实验验证了JPSH是否在相似性搜索任务上优于几个哈希算法。
translated by 谷歌翻译
最近,在大型缩放因素下,单图像超分辨率(SR)通过将预训练的生成对抗网络(GAN)作为先验,见证了令人印象深刻的进步。但是,大多数基于GAN的SR方法受到倒置潜在代码中的属性分离问题的约束,该属性直接导致发电机层中的视觉属性不匹配并进一步退化重建。此外,将馈送给发电机的随机噪声用于无条件的细节生成,这往往会产生不忠的细节,从而损害了生成的SR图像的忠诚度。我们设计了Laren,这是一种潜在的多关系推理技术,可以通过潜在空间中的基于图的多关系推理来实现出色的大型SR。 Laren由两种创新设计组成。第一个是基于图的分离,该解散通过层次多相关推理构建了较高的分离潜在空间。第二个是基于图形的代码生成,该代码生成通过递归关系推理逐渐生成特定于图像的代码,这使先前的gans能够生成理想的图像详细信息。广泛的实验表明,Laren实现了优越的大型图像SR,并且在多个基准测试中始终如一地超过最先进的方法。
translated by 谷歌翻译
神经辐射场(NERF)通过从多视图2D图像中隐式建模3D表示,在新型视图合成中表现出非常令人印象深刻的性能。但是,大多数现有的研究都使用合理的相机姿势初始化或手动制作的摄像头分布来训练NERF模型,这些分布通常不可用或在各种真实世界中很难获取。我们设计了VMRF,这是一种匹配NERF的创新视图,可以进行有效的NERF培训,而无需在相机姿势或相机姿势分布中进行先验知识。 VMRF引入了视图匹配方案,该方案利用了不平衡的最佳传输来制定功能传输计划,以映射带有随机初始化的摄像头姿势的渲染图像,以映射到相应的真实图像。通过功能传输计划作为指导,设计了一种新颖的姿势校准技术,可以通过预测两对渲染图像和真实图像之间的相对姿势转换来纠正最初的随机摄像头姿势。对许多合成数据集进行的广泛实验表明,所提出的VMRF的性能优于最先进的质量和定量,这是大幅度的。
translated by 谷歌翻译
零拍摄对象检测(ZSD),将传统检测模型扩展到检测来自Unseen类别的对象的任务,已成为计算机视觉中的新挑战。大多数现有方法通过严格的映射传输策略来解决ZSD任务,这可能导致次优ZSD结果:1)这些模型的学习过程忽略了可用的看不见的类信息,因此可以轻松地偏向所看到的类别; 2)原始视觉特征空间并不合适,缺乏歧视信息。为解决这些问题,我们开发了一种用于ZSD的新型语义引导的对比网络,命名为Contrastzsd,一种检测框架首先将对比学习机制带入零拍摄检测的领域。特别地,对比度包括两个语义导向的对比学学习子网,其分别与区域类别和区域区域对之间形成对比。成对对比度任务利用从地面真理标签和预定义的类相似性分布派生的附加监督信号。在那些明确的语义监督的指导下,模型可以了解更多关于看不见的类别的知识,以避免看到概念的偏见问题,同时优化视觉功能的数据结构,以更好地辨别更好的视觉语义对齐。广泛的实验是在ZSD,即Pascal VOC和MS Coco的两个流行基准上进行的。结果表明,我们的方法优于ZSD和广义ZSD任务的先前最先进的。
translated by 谷歌翻译
In this paper, we propose a robust 3D detector, named Cross Modal Transformer (CMT), for end-to-end 3D multi-modal detection. Without explicit view transformation, CMT takes the image and point clouds tokens as inputs and directly outputs accurate 3D bounding boxes. The spatial alignment of multi-modal tokens is performed implicitly, by encoding the 3D points into multi-modal features. The core design of CMT is quite simple while its performance is impressive. CMT obtains 73.0% NDS on nuScenes benchmark. Moreover, CMT has a strong robustness even if the LiDAR is missing. Code will be released at https://github.com/junjie18/CMT.
translated by 谷歌翻译
Knowledge graphs (KG) have served as the key component of various natural language processing applications. Commonsense knowledge graphs (CKG) are a special type of KG, where entities and relations are composed of free-form text. However, previous works in KG completion and CKG completion suffer from long-tail relations and newly-added relations which do not have many know triples for training. In light of this, few-shot KG completion (FKGC), which requires the strengths of graph representation learning and few-shot learning, has been proposed to challenge the problem of limited annotated data. In this paper, we comprehensively survey previous attempts on such tasks in the form of a series of methods and applications. Specifically, we first introduce FKGC challenges, commonly used KGs, and CKGs. Then we systematically categorize and summarize existing works in terms of the type of KGs and the methods. Finally, we present applications of FKGC models on prediction tasks in different areas and share our thoughts on future research directions of FKGC.
translated by 谷歌翻译
Few Shot Instance Segmentation (FSIS) requires models to detect and segment novel classes with limited several support examples. In this work, we explore a simple yet unified solution for FSIS as well as its incremental variants, and introduce a new framework named Reference Twice (RefT) to fully explore the relationship between support/query features based on a Transformer-like framework. Our key insights are two folds: Firstly, with the aid of support masks, we can generate dynamic class centers more appropriately to re-weight query features. Secondly, we find that support object queries have already encoded key factors after base training. In this way, the query features can be enhanced twice from two aspects, i.e., feature-level and instance-level. In particular, we firstly design a mask-based dynamic weighting module to enhance support features and then propose to link object queries for better calibration via cross-attention. After the above steps, the novel classes can be improved significantly over our strong baseline. Additionally, our new framework can be easily extended to incremental FSIS with minor modification. When benchmarking results on the COCO dataset for FSIS, gFSIS, and iFSIS settings, our method achieves a competitive performance compared to existing approaches across different shots, e.g., we boost nAP by noticeable +8.2/+9.4 over the current state-of-the-art FSIS method for 10/30-shot. We further demonstrate the superiority of our approach on Few Shot Object Detection. Code and model will be available.
translated by 谷歌翻译
Graph Neural Networks (GNNs) have shown satisfying performance on various graph learning tasks. To achieve better fitting capability, most GNNs are with a large number of parameters, which makes these GNNs computationally expensive. Therefore, it is difficult to deploy them onto edge devices with scarce computational resources, e.g., mobile phones and wearable smart devices. Knowledge Distillation (KD) is a common solution to compress GNNs, where a light-weighted model (i.e., the student model) is encouraged to mimic the behavior of a computationally expensive GNN (i.e., the teacher GNN model). Nevertheless, most existing GNN-based KD methods lack fairness consideration. As a consequence, the student model usually inherits and even exaggerates the bias from the teacher GNN. To handle such a problem, we take initial steps towards fair knowledge distillation for GNNs. Specifically, we first formulate a novel problem of fair knowledge distillation for GNN-based teacher-student frameworks. Then we propose a principled framework named RELIANT to mitigate the bias exhibited by the student model. Notably, the design of RELIANT is decoupled from any specific teacher and student model structures, and thus can be easily adapted to various GNN-based KD frameworks. We perform extensive experiments on multiple real-world datasets, which corroborates that RELIANT achieves less biased GNN knowledge distillation while maintaining high prediction utility.
translated by 谷歌翻译
This paper focuses on designing efficient models with low parameters and FLOPs for dense predictions. Even though CNN-based lightweight methods have achieved stunning results after years of research, trading-off model accuracy and constrained resources still need further improvements. This work rethinks the essential unity of efficient Inverted Residual Block in MobileNetv2 and effective Transformer in ViT, inductively abstracting a general concept of Meta-Mobile Block, and we argue that the specific instantiation is very important to model performance though sharing the same framework. Motivated by this phenomenon, we deduce a simple yet efficient modern \textbf{I}nverted \textbf{R}esidual \textbf{M}obile \textbf{B}lock (iRMB) for mobile applications, which absorbs CNN-like efficiency to model short-distance dependency and Transformer-like dynamic modeling capability to learn long-distance interactions. Furthermore, we design a ResNet-like 4-phase \textbf{E}fficient \textbf{MO}del (EMO) based only on a series of iRMBs for dense applications. Massive experiments on ImageNet-1K, COCO2017, and ADE20K benchmarks demonstrate the superiority of our EMO over state-of-the-art methods, \eg, our EMO-1M/2M/5M achieve 71.5, 75.1, and 78.4 Top-1 that surpass \textbf{SoTA} CNN-/Transformer-based models, while trading-off the model accuracy and efficiency well.
translated by 谷歌翻译